A More General Robust Loss Function

We present a loss function which can be viewed as a generalization of many popular loss functions used in robust statistics: the Cauchy/Lorentzian, Welsch, and generalized Charbonnier loss functions (and by transitivity the L2, L1, L1-L2, and pseudo-Huber/Charbonnier loss functions). We describe and visualize this loss, and document several of its useful properties. Many problems in statistics [8] and optimization [6] require robustness — that a model be insensitive to outliers. This idea is often used in parameter estimation tasks, where a non-robust loss function such as the L2 norm is replaced with some most robust alternative in the face of non-Gaussian noise. Practitioners, especially in the image processing and computer vision literature, have developed a large collection of different robust loss functions with different parametrizations and properties (some of which are summarized well in [2, 13]). These loss functions are often used within gradient-descent or second-order methods, or as part of M-estimation or some more specialized optimization approach. Unless the optimization strategy is co-designed with the loss being minimized, these losses are often “plug and play”: only a loss and its gradient is necessary to integrate a new loss function into an existing system. When designing new models or experimenting with different design choices, practitioners often swap in different loss functions to see how they behave. In this paper we present a single loss function that is a superset of many of these common loss functions. A single continuous-valued parameter in our loss function can be set such our loss is exactly equal to several traditional loss functions, but can also be tuned arbitrarily to model a wider family of loss functions. As as result, this loss may be useful to practitioners wishing to easily and continuously explore a wide variety of robust loss functions.